Method for achieving end-to-end quality of service negotiations for distributed multi-media applications

ABSTRACT

This invention presents a framework for achieving dynamic End-to End QoS negotiation and control coordination, with distributed multimedia applications. The framework builds upon dynamic capability negotiations and specification of Adaptation Paths and (alternative) QoS Contracts, based on user preferences. In particular we present a protocol providing End-to-End negotiation of alternative QoS, capabilities, and preferences/configurations, based on extensions of IP-based protocols like SIP/RTSP/SDP, in coordination with mechanisms for network resource reservation (e.g. RSVP), local terminal resource (e.g. CPU, memory, power, auxiliary devices) reservation, and adaptation mechanisms. To this extent, and with respect to two or more peers ( 101, 103 ) this invention identifies six phases, through which said peers can establish multiparty, multi-stream, multimedia communications. In detail, the phases are: Protocol Discovery ( 104 ), Pre-Negotiation ( 106 ), Multi-Stream QoS Synchronization and QoS Correlation ( 107 ), Fast-Negotiation (obeying the Economy Principle) ( 108 ), Re-Negotiation (obeying the Economy Principle) ( 109 ), Resource Reservation Release ( 110 ). All the six phases can be concatenated, or be executed at different times. This invention also presents the concept of the E2ENP Broker ( 105 ), an optional third-party entity, which can be used for relieving peers ( 101, 103 ) from performing the time- and resource-consuming Pre-Negotiation phase ( 106 ) (and eventually also the Multi-Stream QoS Synchronization and QoS Correlation ( 107 ). This entity may coincide with e.g. audio-/videoconference bridges.

[0001] The invention hereby presented generally relates to the field ofdistributed multimedia applications and technologies. It includesresearch and development issues related especially to multimediamiddleware, Quality of Service (QoS) Management, Resource Reservationmechanisms, mobile terminals, and wireless communication.

[0002] At first the terminology used in the present description and theclaims will be shortly explained: Term Definition Adaptation An orderedset of QoS contracts that indicate the users Path preferences andwishes. Typically, the most important QoS contract is the first one inthe path. Component A particular type of interaction (e.g. audioconversation) that can be realized using different applicationsConfiguration A set of parameters required for implementing a variationof a certain component. Potential configuration indicates the system'sfunctional capabilities, whereas actual configurations reflect the modeof operation of a particular instantiation. Content Exchange ofinformation leading to select proper negotiation representation whentransferring data Economy The economy principle suggests to reserve themore Principle expensive resources as the last ones. As networkresources are shared among several clients and typically one has to payfor them, it is better to first reserve resources on all end systems andthen resource network resources as the last step. Feature set Some setsof media features described by a media feature assertion Quality of Thecollective effect of service performance which Service, QoS determinesthe degree of satisfaction of a user of the service¹ QoS Change Thechange of the QoS contract initiated by the service user QoS ContractAgreement between a service user and a given service provider,specifying QoS requirements and constraints, as well as the policiesrequired to keep track about QoS during all phases of said service. QoSContract Captures the structure of a class of QoS Contracts, by Typeidentifying how individual QoS Contracts specify the QoS properties overa given set of QoS parameter types (also known as dimensions in[Frolu98]). Each parameter type consists of a name and a domain ofvalues. QoS specifications can be simply intended as a set ofconstraints over said domains, one per parameter type. QoS General termfor identifying set of QoS parameters and Specification constraintsspecified by a user. QoS Violation The violation of a QoS Contractcaused by the Service Provider Third party call In third party callcontrol, an external controller is control creating, modifying andterminating sessions between other participants.

[0003] Now the abbreviations used in the description will be explained:ATM Asynchronous Transfer Mode CC/PP Composite Capabilities/PreferenceProfiles CSCW Computer-Supported Cooperative Work E2ENP End-to-EndNegotiation Protocol HDTV High Definition Television HTTP Hyper TextTransport Protocol IETF Internet Engineering Task Force IP InternetProtocol IRT Initiator-Role-Token MSC Message Sequence Charts OSOperating System RDF Resource Description Framework RTP Real TimeProtocol RTSP Real Time Streaming Protocol RSVP Resource ReservationProtocol SAP Session Announcement Protocol SDP Session DescriptionProtocol SIP Session Initiation Protocol VoD Video on Demand UML UnifiedModeling Language XML Extended Markup Language

[0004] The following prior art documents are known to the inventors:

[0005] [Beser00] B. Beser, Codec Capabilities Attribute for SDP, JETFInternet-Draft, Work-in-progress,<draft-beser-mmusic-capabilities-00.txt>.

[0006] [Bhatt99] S. Bhatti, G. Knight, Enabling QoS adaptation decisionsfor Internet applications, London, UK, 1999.

[0007] [Blake98] S. Blake, D. Black, M. Carlson, E. Davies, Z. Wang, W.Weiss, An Architecture for Differentiated Services, IETF Request forComments: 2475, December 1998.

[0008] [Booch99] G. Booch, J. Rumbaugh, I. Jacobson, The UnifiedModeling Language User Guide, Addison Wesley Longman, 1999.

[0009] [Bor00] C. Bormann et. al., Simple Conference Control Protocol,IETF Internet-Draft, Work-in-progress, <draft-ietf-mmusic-sccp-01.txt>.

[0010] [Burn00] L. Burness et al., The BRAIN Quality of ServiceArchitecture for Adaptable Services with Mobility Support, inProceedings of the PIMRC 2000, London, 2000.

[0011] [Cama00] G. Camarillo, Confirmation of SDP preconditions, IETFInternet Draft, Work-in-progress,<draft-camarillo-manyfolks-confirm-02.txt>.

[0012] [Cama00a] G. Camarillo, Third party call control with SDPpreconditions, IETF Internet Draft, Work-in-progress,<draft-camarillo-3pcc-qos-00.txt>.

[0013] [Conneg00] G. Klyne (ed.), A revided media feature set matchingalgorithm, IETF media feature registration WG, Work-in-progress,<draft-klyne-conneg-feature-match-02.txt>.

[0014] [Conneg00a] G. Klyne (ed.), Identifying composite media features,IETF conneg working group, Work-in-progress,<draft-ietf-conneg-feature-hash-05.txt>.

[0015] [Frolu98] S. Frolund, J. Koistinien, QML: A Language for Qualityof Service Specification, HP-Lab Technical Reports, HPL-98-10, 980210.

[0016] [Handl98] M. Handley, V. Jacobson, SDP: Session DescriptionProtocol, IETF Request for Comments: 2327, April 1998.

[0017] [Handl99]M. Handley, H. Schulzrinne, E. Schooler, J. Rosenberg,SIP: Session Initiation Protocol, IETF Request for Comments: 2543, March1999.

[0018] [Handl00] M. Handley, C. Perkins, E. Whelan, Session AnnouncementProtocol, IETF Request for Comments: 2974, October 2000.

[0019] [ISOX641] ITU-T Recommendation X.641 (12/97) |ISO/IEC 13236:1998,Information technology—Quality of Service: Framework.

[0020] [Kassl199a] A. Kassler, P. Schulthess, An End-to-End Quality ofService Management Architecture for Wireless ATM networks, in:Proceedings of the HICSS 32, Mauii, Hi., USA, January 1999.

[0021] [Kassl199b] A. Kassler, P. Schulthess, Extending the QoS Paradigmof Wireless ATM to Mobile Broadband Applications and to the User, in:Proceedings of the WCNC '99, September 1999, New Orleans, USA.

[0022] [Kassl100] A. Kassler et al., BRENTA—Supporting Mobility andQuality of Service for Adaptable Multimedia Communication, inProceedings of the IST Mobile Communications Summit 2000, Galway,Ireland, October 2000, pp. 403-408.

[0023] [Kassl100a] A. Kassler et al., An Open Endsystem Architecture forAdaptable Multimedia Services with QoS Support, in Proceedings of theBRAIN workshop, London, 2000.

[0024] [Levin01] O. Levin, SIP Requirements for support of Multimediaand Video, IETF Internet-Draft, Work-in-progress,<draft-levin-sip-for-video-00.txt>

[0025] [Manda00a] D. Mandato, A universal QoS adaptation framework formobile multimedia applications, European Patent Application 00 111191.3, May 24, 2000.

[0026] [Manda00b] D. Mandato et at, High-Level interface for QoS-basedmobile multimedia applications, European Patent Application 00 126975.2, Dec. 8, 2000.

[0027] [Marsh00] W. Marshall et al., SIP Extensions for ResourceManagement, IETF draft <draft-ietf-sip-manyfolks-resource-00>, November2000.

[0028] [Nahr95] K. Nahrstedt and J. M. Smith, The QoS Broker IBEEE,Multimedia Magazine, Spring 1995 (2)1, pp. 53-67

[0029] [Ott99] J. Ott et. al., Capability description for groupcooperation, IFTF Internet-Draft, Work-in-progress,<draft-ott-mmusic-cap-00.txt>.

[0030] [Pan99] P. Pan, H. Schulzrinne, YESSIR: A Simple ReservationMechanism for the Internet, Computer Communications Review (CCR), Vol.29, No. 2, pp. 89-101, April 1999.

[0031] [Pasqu92] J. Pasquale, G. Polyzos, E. Anderson, V. Kompella, TheMultimedia Multicast Channel, Proc. of 3rd International Workshop onNetwork and Operating System Support for Digital Audio and Video(NOSSDAV 92), San Diego, Calif., November 1992, pp. 185-192.

[0032] [Pasqu93] J. Pasquale et al., Filter Propagation in DisseminationTrees: Trading Off Bandwidth and Processing in Continuous MediaNetworks, Proc. of NOSSDAV 93, Lancaster University, Lancaster, UK,1993, pp. 269-278.

[0033] [RFC2533] RFC 2533, A Syntax for Describing Media Feature Sets,Graham Klyne, 5 GM/Content Technologies, March 1999.

[0034] [RFC2703] RFC 2703, Protocol-independent Content NegotiationFramework, Graham Klyne, 5 GM/Content Technologies, September 1999.

[0035] [SIPRES01] W. Marshall et. al., Integration of ResourceManagement and SIP—SIP Extensions for Resource Management, IETF SIPworking group, Work-in-progress,<draft-ietf-sip-manyfolks-resource-01.txt>.

[0036] (SDPCN00) F. Andreasen, SDP Simple Capability Negotiation, IETFMMUSIC working group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-00.txt>.

[0037] [SDPCN01] F. Andreasen, SDP Simple Capability NegotiationRequirements, IETF MMUSIC working group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-reqts-00.txt>.

[0038] [SDPNG00] D. Kutscher et. al., Requirements for SessionDescription and Capability Negotiation, IETF Internet-Draft,Work-in-progress, <draft-kutscher-mmusic-sdpng-req-01—.txt>.

[0039] [Shulz98] H. Schulzrinne, A. Rao, R. Lanphier, “Real TimeStreaming Protocol (RTSP)”, IETF Request for Comments: 2326, April 1998.

[0040] [SIPRES01] W. Marshall et. al., Integration of ResourceManagement and SIP—SIP Extensions for Resource Management, IETF SIPworking group, Work-in-progress,<draft-ietf-sip-manyfolks-resource-01.txt>.

[0041] [SMIL98] Synchronized Multimedia Integration Language (SMIL) 1.0Specification, W3C Recommendation, Jun. 15, 1998.

[0042] [Yeado93] N. Yeadon, F. Garcia, D. Hutchinson, D. Shepherd,Filters: QoS Support Mechanisms for Multipeer Communications, IEEEJournal on Selected Areas in Communications, Vol. 14, No., 7, September1993.

[0043] The relevant prior art documents will now be acknowledgedshortly:

[0044] In W. Marshall et. al.: Integration of Resource Management andSIP—SIP Extensions for Resource Management, IETF SIP working group,Work-in-progress, <draft-ietf-sip-manyfolks-resource-01.txt>, theauthors present a multi-phase call-setup mechanism that makes networkQoS and security establishment a precondition to sessions initiated bythe Session Initiation Protocol (SIP) and described by the SessionDescription Protocol (SDP). Network resources are reserved before thesession is started using existing network resource reservationmechanisms (like RSVP). The resource management protocol is interleavedbetween two phases of call signaling and participants are invited afterresources are available in the network. A confirmation of thepreconditions is explicitly signaled. Resource management is done onlyfor the network resources. An extension to SDP is introduced todetermine whether the preconditions are met.

[0045] In G. Camarillo: Confirmation of SDP preconditions, IETF InternetDraft, Work-in-progress, <draft-camarillo-manyfolks-confirm-02.txt>, anadditional direction attribute is introduced to indicate which partysends the confirmation of the preconditions. Finally, G. Camarillo:Confirmation of SDP preconditions, IETF Internet Draft,Work-in-progress, <draft-camarillo-manyfolks-confirm-02.txt>provides amechanism to perform third party call control in SIP when SDPpreconditions are used.

[0046] In RFC 2533, A Syntax for Describing Media Feature Sets, GrahamKlyne, 5 GM/Content Technologies, March 1999 the authors present aformat to express media feature sets that represent media handlingcapabilities. In addition, an algorithm is provided that matches thefeature sets. It might be used to determine if the capabilities of asender and receiver are compatible.

[0047] This matching algorithm is improved in G. Klyne (ed.): A revisedmedia feature set matching algorithm, IETF media feature registrationWG, Work-in-progress, <draft-klyne-conneg-feature-match-02.txt>.

[0048] In addition, G. Klyne (ed.): Identifying composite mediafeatures, IETF conneg working group, Work-in-progress,<draft-ietf-conneg-feature-hash-05.txt> describes an abbreviated formatfor composite media feature sets that uses a hash of the featurerepresentation to describe the composite. This can be used to provide anabbreviated form for referencing an arbitrary feature set expression,independent of any particular mechanism for dereferencing. In F.Andreasen, SDP Simple Capability Negotiation Requirements, IETF MMUSICworking group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-reqts-00.txt> the authors state therequirement that a capability set should contain a handle (similar tothe hash mentioned above) allowing for easy referencing of thecapability set.

[0049] In RFC 2703, Protocol-independent Content Negotiation Framework,Graham Klyne, 5 GM/Content Technologies, September 1999 the authorspresent an abstract framework for protocol independent contentnegotiation for the resources with which it interacts. It does howevernot provide the content negotiation process. It identifies the need forexpressing the capabilities of the sender and the data resource to betransmitted, the capabilities of the receiver and the need for aprotocol to exchange these capabilities. Negotiation is carried out by aseries of negotiation metadata exchanges. Negotiation stops, if aspecific data file has been found to be transmitted. The sendertransfers the data if the sender determines the file, otherwise thereceiver informs the sender. This proposal therefore relates to contentnegotiation.

[0050] Work going on in D. Kutscher et. al.: Requirements for SessionDescription and Capability Negotiation, IETF Internet-Draft,Work-in-progress, <draft-kutscher-mmusic-sdpng-req-01.txt> identifiesthe requirements of a framework dealing with session description andendpoint capability negotiation in multiparty multimedia conferencingscenarios. Depending on user preferences, system capabilities or otherconstraints, different configurations can be chosen for the conference.The need of a negotiation process among the peers is identified, but notdescribed, in order to determine a common set of potentialconfigurations and to select one of the common configurations to be usedfor information exchange. This capability negotiation is used to get avalid session description compatible with the end system capabilitiesand user preferences of the potential participants. Differentnegotiation policies may be used to reflect different conference types.They also identify the need for network resource reservation coupledwith session setup. Finally a proposal is drafted for describingcapabilities and providing the negotiation language but not a protocol.

[0051] The authors of F. Andreasen, SDP Simple Capability Negotiation,IETF MMUSIC working group, Work-in-progress,<draft-andreasen-mmusic-sdp-simcap-00.txt> propose a set of SDPextensions providing a minimal and backward compatible capabilitynegotiation mechanism. In B. Beser: Codec Capabilities Attribute forSDP, IETF Internet-Draft, Work-in-progress,<draft-beser-mmusic-capabilities-00.txt> the author extends SDP so thatend-points know the codec choices and can agree on a common set. Thecommunication partner can thus obtain the originators capabilities andpreferences. In J. Ott et. al.: Capability description for groupcooperation, IETF Internet-Draft, Work-in-progress,<draft-ott-mmusic-cap-00.txt> a notation for describing potential andspecific configurations of end systems in multiparty collaborationsessions is given. This enables mechanisms to define end systemcapabilities, calculate a set of common capabilities and to express aselected media description for use in session descriptions. They do notprovide a protocol for capability exchange. In C. Bormann et. al.:Simple Conference Control Protocol, IETF Internet-Draft,Work-in-progress, <draft-ietf-mmusic-sccp-01.txt> the authors defineservices for a simple conference control protocol (SCCP) for tightlycoupled conferences. Member management, application/session managementand access control rules for distributed application resources aredefined. The conference's state, which might be established using SIP,is managed during the lifetime using SCCP. This includes the finding ofappropriate configurations, negotiating for configurations and changingthe configuration. However, no interaction with local resourcemanagement is intended.

[0052] The authors of K. Nahrstedt and J. M. Smith: The QoS Broker, IEEEMultimedia Magazine, Spring 1995 (2)1, pp. 53-67 presented a model foran end-point architecture based on a QoS Broker, which is a functionalentity that orchestrates resources at the end-points and coordinatesresource management across layers. In order to configure the systemproperly, the broker uses admission control and negotiation. Negotiationamong peer entities leads to a valid configuration, which involves allnecessary components of the communication system.

[0053] In the present document the term economy principle is used todescribe the order of reservation described in K. Nahrstedt and J. M.Smith: The QoS Broker, IEEE Multimedia Magazine, Spring 1995 (2)1, pp.53-67:

[0054] First, local resources are reserved.

[0055] Then negotiation with the peer entity leads to a configurationthat can be mapped to resource requirements at the peer, which are thenreserved.

[0056] Finally, reservation of network resources is done in the laststep, because network resources are expensive and shared among multipleusers.

[0057] The authors of O. Levin, SIP Requirements for support ofMultimedia and Video, IETF Internet-Draft, Work-in-progress,<draft-levin-sip-for-video-00.txt> present a set of requirements for acall-control protocol for real-time multimedia support over IP.Capabilities have to be expressed, capabilities have to be signaled toidentify the amount of resources that are necessary, and a call controlmechanisms is needed to open/close/modify media streams within theboundaries set forth by capabilities and reserved resources. Also theypropose to announce new capabilities (if available) during a session. Inaddition, the peers have to agree on a common set of codecs to be used.A session control mechanism to start/stop the streams is put as arequirement.

[0058] In Burness L. et al., The BRAIN Quality of Service Architecturefor Adaptable Services with Mobility Support, in Proceedings of thePIMRC 2000, London, 2000; Kassler A. et al., BRENTA—Supporting Mobilityand Quality of Service for Adaptable Multimedia Communication, inProceedings of the IST Mobile Communications Summit 2000, Galway,Ireland, October 2000, pp. 403-408; and Kassler A. et al., An OpenEndsystem Architecture for Adaptable Multimedia Services with QoSSupport, in Proceedings of the BRAIN workshop, London, 2000, anend-system architecture is presented that integrates local, peer andnetwork resource reservation into a framework for end-to-end QoSmanagement. User Preferences and adaptation paths are used together withQoS states to negotiate QoS at application level. Interaction with localresource management is introduced. The layered architecture providessupport for different types of applications.

[0059] The basic idea of extending existing session protocol layers,like SIP, for pre-negotiating Adaptation Paths at multiple levels ofstream aggregations, is contained in European Patent Application 00 126975.2, December 8, 2000. However, this European patent application doesnot detail a full-fledged End-to-End QoS Negotiation Protocol, and doesnot integrate admission control and resource reservation aspects.

[0060] In the European Patent Application 00 111 191.3, May 24, 2000, adata model detailing ways to achieve QoS synchronization and QoScorrelation among multiple multimedia streams is presented. Such datamodel refines the concepts presented in the aforementioned EuropeanPatent Application 00 126 975.2. Such a data model, however, does notcover the description of capability like codecs.

[0061] Problem to be Solved by the Invention

[0062] In [Levin01], the author states on page 2: “( . . . )establishing a session of a certain quality, based on capabilitiesagreement during the session establishment and sustained throughout theduration of the session. The problem can be described as a lack ofexpressiveness in three following areas: capabilities specification,resource reservation and media stream control.

[0063] The desired ultimate goal is

[0064] to express the capabilities (i.e. supported media, CODECalgorithms, bandwidth, etc.) without a need in configuration,

[0065] to signal the total resources required for a specific session(probably in terms of capabilities, and

[0066] within a session, to explicitly open and close media streams andto modify the parameters of a certain stream within the boundaries ofpreviously announced capabilities and reserved resources.”

[0067] The author also states on page 4: “SIP extensions may be neededto separate the ‘capabilities announcement’ phase from the actualopening of media streams.”

[0068] In view of the prior art it is the objective of the presentinvention to propose a concept providing QoS guarantees in an effectivemanner.

[0069] This objective is achieved my means of the features of theindependent claims. The dependent claims develop further the centralidea of the present invention.

[0070] Further features, objects and advantages of the present inventionwill now be explained with reference to the figures of the encloseddrawings:

[0071]FIG. 1 shows high-level MSC identifying phases,

[0072]FIG. 2 shows a functional decomposition of a structure toimplement the present invention,

[0073]FIG. 3 shows an example of a pre-negotiation phase 106 (MSC),

[0074]FIG. 4 shows an example of a fast negotiation phase 108 (MSC), and

[0075]FIG. 5 shows an example for the implementation of an E2ENP Brokerin case of SIP implementation.

[0076] Following are those requirements which can be of importance forproviding QoS guarantees in an effective manner:

[0077] Provision of a comprehensive set of mechanisms and protocols,linking together several aspects like:

[0078] 1. Pre-Negotiation, which also can be done offline and publishedusing e.g. LDAP

[0079] 2. Negotiation integrated with ad-hoc reservation

[0080] 2.1 Economy principle

[0081] 2.2 Coordination of resource reservation, enhanced with thefeatures listed below

[0082] Specific description of the interaction with local and peerresource management (some protocol procedures are required to thisextent)

[0083] Use of state IDs to construct and reference an Adaptation Path

[0084] Explicit mention to hierarchical state machines for AdaptationPath at different levels

[0085] One of the targets the invention is aiming at is the support ofvarious types of services:

[0086] Conversational services

[0087] Distribution services

[0088] Information retrieval services

[0089] The concept proposed by the present invention can be adapted tothese different types of services. The reason being that, depending onthe type of service, different sequences of actions may be required forthe interaction between reserving network and end system resources,which also depends on the communication mode of each peer(sender/receiver).

[0090] For instance, questions need to be addressed, like whether areceiver needs to wait to be contacted by a sender, or who initiatesresource reservation on the network (the sender or the receiver). Theanswers to these questions depend on the type of service.

[0091] The Concept of an End-to-End Negotiation Protocol (E2ENP)

[0092] The establishment of a QoS-enabled communication session can beaccomplished in a multi-step process, starting with negotiation of QoSaspects on an end-to-end basis. The idea hereby proposed is to havepeers negotiating beforehand (i.e. before the actual communication takesplace) a common level of QoS, the use of which peers can agree upon.This is a form of pre-negotiation of a vocabulary which peers can uselater on for efficiently dealing with contingent negotiations (e.g. whenestablishing audio/video streams) or re-negotiations (changing the QoSContract during ongoing sessions). The benefit of this approach is thatthe time necessary for re-negotiation is reduced because the peers haveonly to refer to a negotiated state instead of performing a fullnegotiation cycle during streaming. Furthermore, the type of negotiationcan be tailored based on capabilities that are currently available atall peers.

[0093] The identified steps are:

[0094] 1. Pre-negotiation of a set of QoS contracts with the peers. Suchset describes an Adaptation Path for a generic stream of a given type(e.g. audio, video, or data streams).

[0095] 2. As an intermediate step, peers negotiate QoS correlation andsynchronization aspects among multiple streams. This might not benecessary if a session contains only one stream: in that case QoScorrelation would be simply ignored. Thus this step builds up thesession concept. Note that, in case peers have to coordinate severalstreams, QoS correlation and synchronization issues would be handled atruntime by triggering specific coordination function (based on e.g.RTP), which are derived from the negotiated QoS states.

[0096] 3. As a last step, peers negotiate QoS Contracts on a per streambasis at stream establishment time, based on the pre-negotiatedvocabulary established during step (1). At this step, actual admissioncontrol is performed. To this extent, according to one aspect of thepresent invention it is proposed to follow an economy principleobserving the fact that network resources have to be shared and are moreexpensive than local resources. The hereby proposed process is based onthe steps in the following order:

[0097] 3.1 Local resources (i.e. resources managed by the local system)like CPU, memory, and power are reserved on the terminal device of theparty initiating the communication (i.e. the initiator). These are thecheapest resources, since they are used exclusively by the initiator.The amount of resources that are reserved by the initiating partycorresponds to the quality the initiator is most interested in and cancope with.

[0098] 3.2 Remote resources like CPU, memory, and power are reserved onthe terminal device of the party accepting the request to establish thecommunication (i.e. the responder). These resources may be eventuallyshared by a plurality of initiators with respect to multipletelecommunication sessions, like in the case of a Video-on-Demandscenario (where the responder is the video server, which needs to servemultiple requests from multiple users). It does not make sense to try toreserve these shared resources (thus affecting other communicationsessions at the responder) prior of having ascertained the availabilityof resources at the initiator side.

[0099] 3.3 Only when both the local and remote resources have beensuccessfully reserved, network resources are reserved which are shared(by definition) by a plurality of parties and therefore to this extentcan be considered as the most expensive resources at all.

[0100] This invention describes a protocol procedure for informing peersabout changes in capability configuration (e.g. the installation orremoval of a given multimedia codec) of a given participant of the givencommunication session. By pro-actively disseminating this informationamong peers, proper actions can be taken with respect to any currentlyactive communication session, and to all the negotiation—processesconcerning future attempts-to-establish-communication sessions.

[0101] Steps (2) and (3) (or (1)², (2), and (3)) can be treated jointlyas one-shot³ or separately as a sequence of actions⁴.

[0102] Note that in certain cases only steps (3.1), (3.2) and (3.3)might be necessary. As an example this can be the case if everything ispre-negotiated by default (using fixed settings) and only one qualityand configuration of a video stream is located on a server.

[0103] A key aspect of the E2ENP is that it can be used not only inconjunction with network resource reservation signaling protocols, butalso with other types of QoS architectures. In the latter case,pre-negotiation will focus on having peers agreeing on the types of QoSclasses to use, with respect to the capabilities of each peer.

[0104] What to Negotiate

[0105] This section focuses on (i) what type of information (ii) underwhich circumstances needs to be negotiated.

[0106] More specifically, peers should agree on:

[0107] which E2ENP (or other protocol) to use;

[0108] which capabilities and configurations are available andapplicable;

[0109] what QoS contracts are applicable and form the Adaptation Path;and

[0110] the Adaptation Paths themselves.

[0111] Types of Supported E2ENP

[0112] Most of the state of the art applications will not be able toemploy the hereby-described E2ENP without any specific retrofitting.Rather they will eventually apply some other form of state of artnegotiation mechanism. It is reasonable to assume also that some futureapplications may eventually only partially support the E2ENP, or justdifferent versions thereof.

[0113] For these reasons, the first type of information that needs to benegotiated is the type and version of supported ,,E2ENP”.

[0114] This might be accomplished by using a directory server. As anexample, a directory entry associated with a given peer, and named E2ENPwould act as a token indicating that the given peer supports the E2ENP.The directory entry could look like:

[0115] where “type of service” could indicate information retrievalservice or conversational service, “protocol” could indicate SIP.(together with SDP) or H.323 or RTSP (i.e. which protocol is the givenE2ENP based upon), and “version” could be a number discriminatingincremental sets of functionality. In addition to the aforementionedprotocols, specific extensions are needed to carry out the negotiationand describe the Adaptation Path.

[0116] As an example, the E2ENP can be mapped—based on the type ofservice—to the following protocols: Type of Service ProtocolsConversational services SIP(together with SDP)/H.323 + enhancementsDistribution services SAP + RTSP + enhancements Audio/Video on DemandRTSP + enhancements (Push/Pull) services

[0117] This information can be not only directly queried on purpose bythe peers, but the directory servers themselves can also pro-activelydisseminate it.

[0118] Capabilities

[0119] Various types of capabilities need to be negotiated among peers:

[0120] Media Types and QoS Contract Dimensions

[0121] Bitrates

[0122] Codecs

[0123] Terminal capabilities

[0124] Network Resource Reservation support

[0125] The following paragraphs analyze in detail each type ofcapabilities.

[0126] Media Types and QoS Contract Dimensions

[0127] The type of media (audio/video/data) can be used for screeningany piece of information in the E2ENP messages. Peers agree on a commonset of media types. As an example, if a receiver requests to establish alive audio stream but the corresponding audio stream sender does nothave a microphone, further negotiation of audio codecs is pointless andtherefore it can be skipped. The use of media types thus translates instructuring the E2ENP messages in a format which is convenient for thepeers while negotiating. In addition to media types, this inventionidentifies the need to include the list of QoS Contract Dimensions whichthe contracts are made of. As an example, one peer might want tonegotiate frame-rate and frame-size, whereas other peers may want tonegotiate frame-rate only.

[0128] As an example, it might be useless to try to negotiateframe-size, if a VoD server has only one representation (version) of themovie available, and there is no network media adaptation unitavailable.

[0129] This invention proposes the following format describing mediatype information:

[0130] The refinements ,,param_audio” and ,,param_video” detail the QoScontract dimensions, like frame-size, frame-rate for video and samplingfrequency and number of channels for audio, and are described in thenext sections. This format is to be applied to each media typedescription in a given E2ENP message.

[0131] Codecs

[0132] Before peers can negotiate QoS Contracts or even Adaptation Paths(see below), they shall agree first on which codecs are generallyapplicable. The QoS achievable depends also on the type of codecsavailable. Furthermore, even though a given peer might be equipped witha given set of codecs, the use of such codecs is actually constrained bythe availability of equivalent codecs by all the other peers.

[0133] Today's SIP implementations allow the negotiation of codecs atinvitation time on a peer-to-peer basis, and with respect to thespecification of well-defined streams. In the framework of the presentinvention an extension of such a SIP capability is proposed, bynegotiating during step (1) of the above cited steps (1) to (4) thecomplete list of available codecs, irrespective of which streams will beused later at stream establishment time. In other words, peers can inthis way agree on a common subset of codecs before the actual mediaspecification takes place.

[0134] Furthermore, since current and future terminal devices shall beable to install or remove codecs ,,on the fly”, the common set of codecsmight need to be later re-negotiated during connection time (e.g. duringa videoconference). The re-negotiation could be achieved by using newSIP messages.

[0135] Bitrates

[0136] Two main mechanisms are employed by clients to minimize theamount of bandwidth used for multimedia applications:

[0137] Using low (constant) bitrate codec thus reducing network resourceconsumption,

[0138] Using variable bitrate codecs

[0139] Clients have to agree on a certain set of compatible codecs. Itmust, however, be assured that the bitrate the codec generates issupported both by the clients (for sending/receiving) and by the network(both the access networks and the core). In addition, one codec maygenerate different bitrate levels, which depend on the codecconfiguration. As an example, the G.726 codec may generate bitrates of16, 24, 23 or 40 kb/s. There may be situations where the receiver isable to handle only a certain configuration of the given codec. As anexample, if the receiver uses a 19.6 kb/s modem, the receiver can onlyhandle the 16 kb/s configuration of the G.726 codec. In addition,certain audio codecs and several video codecs can generate a variablebitrate, which is typically specified using a leaky bucket model. Inthis case the bitrate requirements are specified using a sustainablebitrate, a peak bitrate and a burst time (time duration that indicateshow long the source is allowed to send at the peak rate).

[0140] This invention proposes that in addition to a negotiation of theset of compatible codecs, for each codec the peers have to agree on avalid codec configuration in terms of bandwidth that each peer canhandle. The initiator provides and negotiates a set of bandwidthspecifications with the peers using the leaky bucket model (sustainablebitrate, peak bitrate, burst time), which supports both constant andvariable bitrates. In the above example, both parties would agree on theG.726 codec with the following bitrate specification: (sustainablebitrate, peak bitrate, burst time)=(16, 16, N/A).

[0141] Having agreed on a compatible codec configuration in terms ofbitrates, actual target bitrates that are being used in the actualconfiguration are negotiated with the network during step (3).

[0142] Terminal Capabilities

[0143] The physical characteristics (e.g. monitor size) of terminaldevices may change from is model to model, and affect the delivery ofservices over telecommunication networks. This is important, as it makesno sense for instance to stream a video in HDTV quality to a mobilephone, which is equipped with a much smaller display. Therefore, peershave to agree on terminal capabilities when negotiating QoS. Suchterminal capabilities are e.g. the size of the display, the number ofcolors available, the general capabilities of rendering a given mediatype (e.g. if an audio mixer is available), and all kind of informationfor services that are necessary for rendering a given media type.

[0144] To this extent, the Composite Capabilities/Preference Profiles[CC/PP] framework language (a combination of XML and RDF) andrequest/response HTTP protocol offer a handy way for specifying terminalcapabilities in an RDF format.

[0145] Network Resource Reservation Support

[0146] It is useless to try to demand network resources end-to-end usinga signaling protocol like RSVP, if it can be determined a priori thatone or multiple remote peers are not RSVP aware. To this extent, apossible solution is to ask peers to check if a given signaling protocolis available at their access networks.

[0147] A less complex solution is that, during pre-negotiation, peersindicate to each other along which direction they are going to reservenetwork resources. Allowable values are N/A, sender, receiver,sender/receiver, where the concepts of sender and receiver relate to theresource reservation protocol used (if any). This information can infact model the type of reservation protocol used, if any.

[0148] For instance, in a simple Video-on-Demand service scenario withRSVP available end-to-end, the client acts as a receiver, whereas theserver acts as a sender, since RSVP is receiver-initiated. In case ofYESSIR (see [Pan99]), which is sender-initiated, the configuration—froma resource reservation protocol perspective—would be the opposite.

[0149] Similarly, in a videoconference scenario with end-to-end RSVPsupport, each peer would be typically configured as a sender/receiver.

[0150] If no direction is specified for a given peer, the correspondingpeers can immediately deduce that no network resource reservationmechanism is available on the given peer. The key point is that thismodel allows to cope also with cases where no QoS signaling protocol(like RSVP) is available end-to-end. In such a case, each peer would besimply modeled as a sender (or nothing at all). This inventionintegrates the latter solution.

[0151] QoS Contract

[0152] QoS Contracts are hereby defined as sets of high-level QoScharacteristics, each expressed in terms of any of the following:

[0153] Operating target: the desired value

[0154] Operating region: a desired interval of values

[0155] Thresholds/limits: a set of values marking different states withrespect to the QoS adaptation business logic.

[0156] Any of the aforementioned QoS specifications can eventually befurthermore expressed in a statistical format, by e.g. statingpercentiles, mean value, variance, etc.

[0157] Adaptation Path

[0158] Peers can agree not only on a given QoS Contract, but also onalternative ones, which can be advantageously used whenever the networkand/or terminal resources availability change over time. In such a way,each peer will exactly know which alternative QoS Contract (and underwhich circumstances) shall be enforced in order to cope with a criticalQoS Change or with any QoS Violation with respect to the currentlyenforced QoS Contract.

[0159] During the step (1) of the above-identified steps (1) to (4) onlystream-related Adaptation Paths are pre-negotiated. Higher level ones(associated with QoS Correlation issues) shall be negotiated at higherlevel at step (2) of the above-identified steps (1) to (4).

[0160] Together with the best-case QoS Contract, the Adaptation Pathforms an extended QoS Contract. This extended QoS Contract is negotiatedand indexed. If a peer later decides to switch to a lower level of QoS,only the index identifying the QoS Contract as a state of a hierarchicalFinite State Machine is needed to tell the other peer to adapt to thatQoS State.

[0161] Types of E2ENP

[0162] The actors participating to communication sessions can act roleslike:

[0163] Sender/Receiver: the Sender can be solely transmitting data to aReceiver, whereas a Receiver receives data from a Sender. A two-wayconference can be modeled with a Sender/Receiver and Receiver/Senderpair.

[0164] Initiator/Responder: the initiator invites responders toparticipate to a communication session. The responder waits for arequest from the initiator.

[0165] Possible combinations are:

[0166] Initiator and Sender→Responder and Receiver

[0167] Initiator and Receiver→Responder and Sender

[0168] In the case where the Initiator is the Sender, the Sender of amedia stream initiates the session (the Sender invites and the Receiverwaits to be contacted). In the case where the Initiator is the Receiver,the Receiver of a media stream initiates the session (the Receiverinvites and the Sender waits to be contacted). An Initiator/Sender cantalk with a Responder/Receiver. An Initiator/Receiver can talk with aResponder/Sender.

[0169] In the following the key scenarios affecting the type of E2ENPconsidered in this invention are listed.

[0170] Peer-to-Peer negotiation: Can be actually extended also tomultiple peers participating to e.g. a videoconference, but where eachpeering is regulated by a direct peer-to-peer QoS negotiation mechanism.

[0171] 1×N Multicast negotiation: In this case, a sender and multiplereceivers need to negotiate one or multiple QoS characteristics at atime. This can be done either on a connection-wide basis (i.e.determining a least common denominator good for all peers) or on areceiver-selected basis (peer-to-peer negotiation for some QoScharacteristics among certain peers—see previous point). In case of abest effort level of agreement, receivers are still able to communicateeven if they cannot agree on a least common QoS level, otherwise (e.g.in case of a guaranteed QoS policy), those receivers must be dropped orother 6 used. As an example, media filters or transcoders can be used inbetween a Sender and a Receiver to tailor media streams in such a way,that Receivers' capabilities and QoS states can be matched with Senders'QoS states. The media transcoder can be used to translate from onerepresentation to another (e.g. switching the codec of the media streamfrom PCM to GSM audio). Media filters can be used for finer granularadaptation of media qualities (e.g. adapting to frame size, frame rateor desired quality without forcing the sender to adapt).

[0172] The connection-wide E2ENP can be effectively used for enforcingQoS Correlation among multiple flows from various sources. The 1×NMulticast negotiation may be used for determining the common QoS notonly for the case of a unique Sender transmitting to multiple Receivers,but also for the case of an unique Receiver correlating the QoS ofincoming flows from many Senders.

[0173] ITU-T Recommendation X.641 (12/97) ISO/IEC 13236:1998,Information technology—Quality of Service: Framework, describing theaforementioned types of negotiations, deals with 3-party negotiation,involving an Initiator, a Provider, and one or multiple Responders. Inthis invention we limit our scope however to the relationship between anInitiator and one or multiple Responders at application level. Thechoice of not directly involving explicitly Providers is due to the factthat Providers are only concerned with traffic and admission control.Therefore, only traffic contracts need to be negotiated between peersand the Network Providers. this can only be done during sessionestablishment, as it does not make sense to reserve resources(especially network ones) for a given period of time in advance, i.e.without using them.

[0174] Peers can therefore perform the hereby-described E2ENPpre-negotiation phase by agreeing upon nominal QoS levels. During actualconnection establishment, peers then use the pre-negotiated contracts.These are tested locally, and resources are reserved. The result of thisstage is then offered to the peers, and any excess resources areeventually released. Finally, at this level, network resources arereserved, which again may lead to release of excess resources. Bymonitoring the actual connection quality, peers can then decide at anytime whether the given pre-negotiated QoS contracts are satisfied. Andif not, whether and how to select an alternate QoS level, based on thepre-negotiated adaptation path. Provisions to avoid frequent requests tochange network resource reservations are put in place by properlyintroducing tolerances and hysteresis in the QoS contracts andadaptation paths, as described in the European Patent Application 00 126975.2 [Manda00b].

[0175] In any case, during the pre-negotiation phase, each peer assessesthe bids received from other peers against pre-configured User-Profiles,which contain pre-computed Adaptation Paths. The assessment can beachieved by comparing each parameter of is each QoS Contract provided inthe bid, with the corresponding information contained in the UserProfiles. The decision whether to accept a bid or formulate acounteroffer is made with respect to all the QoS Dimensions of the bidQoS Contracts. Should the bid propose QoS Contracts with different QoSDimensions, only the subset of mutually understood QoS Dimensions wouldbe taken into account.

[0176] The initiator associates each QoS Contract of its bid with anidentifier. Responders will notify the acceptance of a given QoSContract (or the proposal for an update thereof, as a counteroffer), byspecifying the corresponding identifier. Should a Responder feature abetter QoS Contract resolution (like in the case of a given bid QoSContract, in correspondence to which multiple QoS Contracts with “finer”granularity, are available at one of the Responders), said Responder mayreturn a counteroffer with new QoS Contracts. It is up to the Initiatorto collect these counteroffers, choose the most meaningful ones,associate them with new identifiers, and propose them to all peers in anew negotiation round.

[0177] Mapping between Type of Service and Type of E2ENP

[0178] The following paragraphs describe how E2ENP can be deployed withrespect to the various types of services mentioned above.

[0179] Conversational Services

[0180] This is the case of services like real-time videoconference orCSCW. The key characteristic of these services is that the type ofcontent exchanged among peers is most likely based on user interactionswith the systems and with other peers. For instance, most of video flowswill carry live images (e.g. taken from a webcam).

[0181] Another important aspect is that peers need to agree beforehandon a given agenda, which includes the start-time and duration of theservice. This is typically accomplished by using SAP and SIP, in theInternet world. The manner how this type of services is actuallydeployed, is highly implementation dependent. Taking a videoconferenceas an example, special bridges are used in the telecom world so as tooffer a star topology solution. Any interested party can thus join avideoconference by simply contacting the bridge. In the Internet worldon the other hand, it is common the use of multicast technology: anyinterested party would simply have to “tune” on the proper multicastgroup. From an E2ENP perspective, however, we hereby assume that peersshall use unicast signaling for negotiating a common level of QoS.

[0182] Conversational services can be modeled as cases of one Initiatorand multiple Responders, where all peers can act as Sender and/orReceiver. Furthermore, from a resource reservation protocol, each partycan use either a sender-driven or a receiver-driven network resourcereservation protocol (if any). Thus, an Initiator requests someresources from the network and from a Responder (since the Responder hasto provide the resources for decoding and displaying). In addition, theInitiator manages its own resources locally.

[0183] Distribution Services

[0184] Distribution services can involve a content provider and (i)known or (ii) unknown content consumers.

[0185] In the former case, SIP or RTSP extension might be used similarlyto the case of information retrieval services (see below), in the lattercase SAP extensions are hereby taken into account. The sender shallsimply announce the list of multicast groups, which the receivers maychoose from for “tuning” on the current content distribution.

[0186] Except for the case of known content consumers, only a reducedversion of E2ENP can be mapped to SAP. The Adaptation Path can be simplymodeled as having the content provider streaming the same content withdifferent QoS levels to different multicast groups (one per QoS level).Content consumers shall therefore simply monitor the level of QoSavailable at their terminal/access network, and eventually tune onanother multicast group if any QoS Change or QoS Violation occurs in themeanwhile. Alternatively, the full-blown E2ENP could be used inconjunction with media filter/transcoder in the network, tailoring themedia streams to match the different QoS levels. To this extent, thecontent consumers should simply instruct the media transcoders/filtersto switch to a new QoS level. It is up to the Content Provider and tothe Receivers to ensure that the streams distributed on differentmulticast groups are mutually synchronized.

[0187] For optimization purposes, the sender may however collectstatistics concerning the number of receivers per multicast group(channel), so as to avoid flooding the network with packets of unusedchannels. This requires signaling between the Content Provider and eachReceiver. This invention however does not address such cases.

[0188] Information Retrieval Services

[0189] This type of services is typically managed by using RTSPprotocol. This invention proposes enhancements of such protocol forachieving E2ENP goals similarly to what described for the case ofconversational services. More specifically, one has to note that theRTSP process is initiated by the receivers by requesting the sender todescribe a given media (e.g. a video stream) through the RTSP DESCRIBEmethod. Through this method, a receiver gets informed about the variouslevels of QoS, which the sender may enforce for playing out the givenmedia. The receiver has thus the possibility to choose which level toenforce at any given time. This is a form of pre-negotiation processpaired with a fast negotiation cycle, similar to the concept presentedby this invention. The key idea is to enhance the TRSP DESCRIBE methodto include all the information about capabilities described above, andto integrate RTSP with the hereby-presented distributed resourcemanagement mechanism.

[0190] QoS Correlation features can also be achieved by using SMIL[SMIL98]. This invention does not address the integration of suchtechnology.

[0191] End-to-End Re-Negotiation Protocol (E2ERP)

[0192] The E2ERP is the subset of the E2ENP focusing on re-negotiationissues. This protocol is activated whenever:

[0193] 1. QoS Violations have been detected at functional entityinvolved in the transmission process; or

[0194] 2. the current monitored level of QoS is no longer compatiblewith the currently enforced QoS Contract; or

[0195] 3. a notification of newly discovered additional resources hasbeen received by some specific functionality; or

[0196] 4. one of the peers decided to enforce a different QoS Contract(e.g. the user requested a higher frame-rate).

[0197] Cases 2 and 3 are mostly related to optimization processes, whichallow applications to upgrade/degrade their QoS requirements in a smoothmanner, whereas case 1 deals with dramatic shortages of resources, whichbadly affect applications, if not properly and promptly treated.

[0198] To this extent, case 1 implies re-negotiating local andeventually network resources without the need to try to maintain thepreviously reserved resources, whereas this is not possible in cases 2and 3. This means that whenever dealing with optimization issues, onehas to take care of:

[0199] either allocating resources in excess of the difference betweenthe previously allocated resources and the new amount (in the case ofQoS upgrade);

[0200] or allocating the new resources from the scratch, while keepingthe old ones until the re-negotiation has been successfully completed(pessimistic approach). This can also be modeled as trying to allocatemore resources while still keeping the old one. If this re-allocationfails, the originally reserved resources are still reserved, otherwisethe newly allocated resources are available;

[0201] releasing some resources to switch to a lower QoS state (in thecase of QoS downgrade).

[0202] The first option seems to be quite attractive, but some resourcesmight not be easily shared among different QoS States. For example, thescheduling information about the thread of a given codec might betotally useless, if the codec is going to be preempted and substitutedwith another one, which runs on a totally different thread.

[0203] Therefore the second approach, though pessimistic, seems to bethe most realistic one. In the following we will therefore referexplicitly to the allocation of new resources and later release of oldones. In addition, releasing resources is always possible, because thesystem load is decreased.

[0204] Whenever a re-negotiation is triggered, each peer determines fromthe pre-negotiated Adaptation Path the best alternative QoS Contractthat should be enforced in order to cope with the detected QoSChange/Violation. The identity of the new QoS Contract is exchangedamong peers, in order to come to an agreement on which QoS Contractshould be actually enforced. Having previously negotiated the AdaptationPath allows the peers to more efficiently perform such re-negotiating,by simply exchanging identifiers. Should all peers detect a violationsimultaneously, some form of contention resolution algorithm would needto be enforced. To this extent different well-known techniques areavailable: assigning master/slave roles (typically assigning to theInitiator the master role), or re-attempting after a random time haselapsed. This invention leverages the master/slave mechanism.

[0205] In summary, the following types of re-negotiation are identified:

[0206] a) re-negotiation-triggered by a QoS violation detection, wherenew resources are allocated from the scratch and the old resources areautomatically released (for QoS downgrade cases, this might be not anissue);

[0207] b) re-negotiation for optimizing the system, where new resourcesare allocated from scratch and the old resources are released later, atE2ERP successful completion; and

[0208] c) re-negotiation for optimizing the system, where new resourcesare allocated from scratch and the old resources are automaticallyreleased.

[0209] A framework for Dynamic End-to-End QoS Negotiation and Control

[0210] This invention presents a framework for achieving dynamicend-to-end QoS negotiation and control coordination, for distributedmultimedia applications. The framework builds upon dynamic capabilitynegotiation and specification of Adaptation Paths and QoS Contracts,based on user preferences. These principles will now be explained withreference to the figures.

[0211] The Initiator 101 finds the protocol version using a ProtocolDiscovery Unit 104, 213 via interface 214, which queries a DirectoryServer 102, 215. This is coordinated with the local Coordination Unit204. The Session Protocol Unit 205 is used for all kind of negotiationsand communication. In particular it negotiates with the Peer 216 (whichis the Responder 103) a common set of capabilities and configurationsand thus builds a vocabulary in the Pre-Negotiation phase 106. It alsonegotiates QoS Contracts and Adaptation Paths that are used for theMulti-Stream QoS synchronization and QoS correlation phase 107. LocalResource management is carried out by the Local Resource ManagementUnits (responsible for CPU, buffer and other local resources) 206 viainterface 211. Local Resource management is coordinated by theCoordination Unit 204 with network resource reservation, which iscarried out by the Network Resource Reservation Unit 207, via interface212.

[0212] Any end-to-end interaction during communication sessionestablishment, i.e. during the Fast Negotiation phase 108, is carriedout by following the Economy Principle. This principle states that firstlocal, then peer and finally network resources are reserved. If duringthe communication session a re-negotiation is necessary due to a changein resource availability or to any other reasons influencing resourceavailability, the Re-Negotiation phase 109 is carried out by followingthe Economy Principle as well.

[0213] Finally, resource reservations release is done during the Releasephase 110. The local Protocol Stack 208 interfaces with the SessionProtocol Unit 205 via the interface 217 and implements the sessionprotocol. In addition, the Protocol Stack 208 implements the networkresource reservation protocol which is controlled via interface 210.

[0214] The QoS-Aware Application Unit 202 uses a FSM Engine Unit 203 tocontrol the states of all the streams that the QoS-Aware ApplicationUnit 202 handles. The Coordination Unit 204 together with FSM EngineUnit via interface 209 provides the QoS-Aware Application Units 202 withthe possibility to establish, use, control and release networkconnections with other applications, using QoS guarantees. Establishingsuch connections involves negotiation of capabilities, configurations,QoS Contracts and Adaptation Paths (managed by the FSM Engine Unit 203),and integration of local, peer, and network resources according to theEconomy Principle. Control involves re-negotiation and integration oflocal, peer, and network resources and may involve a state transitionmanaged by the FSM Engine Unit 203, in cooperation with the CoordinationUnit 204. Release involves network and local, as well as peer resourcetear-down and state management (by the FSM Engine Unit 203), incooperation with the Coordination Unit 204.

[0215] This invention also presents the concept of the E2ENP Broker 105,an optional third-party entity, which can be used for relieving peers101, 103 from performing the time- and resource-consumingPre-Negotiation phase 106 (and eventually also the Multi-Stream QoSSynchronization and QoS Correlation 107). Each peer 101, 103, afterdiscovering from a Directory Server 102 which E2ENP Broker 105 tocontact, can upload its Adaptation Path, Capabilities, and the list ofpeers to contact on said E2ENP Broker 105. The E2ENP Broker 105 may beimplemented e.g. as an enhanced version of audio-/videoconferencebridges, or by combining an enhanced SIP proxy server (or SIP redirectserver)—dealing with the negotiation process—and an enhanced version ofa SIP registrar—storing the Adaptation Path and Capabilities and all theintermediate negotiated versions thereof.

[0216] The E2ENP Broker 105 thus collects such information from eachpeer 101, 103, and performs the negotiation process locally in itsmemory space. During this process, the E2ENP Broker 105 may eventuallycontact some of the indicated responders 103 on behalf of theinitiator(s) 101, should such responders 103 do not have yet storedtheir information on said E2ENP Broker 105. Once the negotiation hasbeen carried out successfully, the E2ENP Broker 105 will notify thepeers 101, 103 about the availability of the results, and each peer 101,103 can then download said results. The E2ENP Broker 105 can also allowmultiple negotiation iterations (otherwise too expensive to be carriedout among peers themselves). Responders 103 might disagree with a bidfrom the Initiator 101, and therefore provide counteroffers. Theanalysis of counteroffers involves multiple round-trip message exchangesamong peers, and therefore is too time consuming. The E2ENP Broker 105can mediate all this process.

EXAMPLE

[0217] The example in this section is illustrating the concepts of ourinvention. It is based on a real-time multimedia conferencing scenariousing IP-based networks. Media transport is carried out by usingRTP/RTCP.

[0218] For simplicity reasons, we are showing the invention's principleusing a pseudo-protocol (QUERY, RESPONSE, COMMIT, START, START_ACK,RESERVE, CONFIRM) which can be mapped onto existing SIP/SDP plusextensions to be standardized. Also, this example is restricted to twopeers A and B. Let us assume that the Initiator 101 wants to receive amedia stream (let us assume video) from the Responder 103.

[0219] The E2ENP process starts with the Pre-Negotiation phase 106assuming that the Protocol Discovery 104 has successfully been completedearlier, figuring out a common E2ENP version that both partiesunderstand.

[0220] In the Pre-Negotiation phase 106, the QoS-Aware Application Unit202 requests-a negotiation using the local FSM Engine Unit 203,providing a bid. This bid contains a list of capabilities andconfigurations as well as QoS Contracts and the Adaptation Path(s),which are all indexed. The description of the bid can be realized usingSDP extensions (e.g. as a new feature of the currently investigatedSDPng [DPNG00]).

[0221] The Coordination Unit 204 forwards the bid via the sessionprotocol unit 205 to the peer using the QUERY message, which can bebased on SIP extensions. The peer's Session Protocol Unit 205 receivesthe QUERY and asks the Peer Coordinator 204 to evaluate the bid usingthe Peer FSM 203. The Peer FSM matches the bid with its own adaptationpath and capabilities and generates a common subset, which is forwardedin the Response massage via the Peer Coordinator 204 to the Peer SessionProtocol Unit 205, which generates a RESPONSE message which can be basedon SIP extensions. The initiator's Session Protocol Unit 205 uponreception of the new bid forwards an evaluate request to the local FSM203 via the Coordinator 204. Finally, the new bid is again checked forcompatibility (it might be empty indicating that the peers areincompatible) and the Initiator's (101) FSM Engine 203 collects all thenew bids from all responders and generates a common subset, which issent to all peers. This is done by first sending an enforce message to204, which forwards it to 205. The initiator's Session Protocol Unit 205generates the commit message, which can be based on SIP extensions. Thismessage carries the bid-match result (which can be described using SDPextensions). This is distributed to the session protocol unit 205 of theresponder 103, which enforces the result via the Coordination Unit 204to the FSM 203. At this point, all communication partners have agreed ona common set of codecs, capabilities, adaptation paths and QoS statesand are able to decode later on the references (denoted as StateIDs).

[0222] The assumption which shall be made now is that the initiator 101wants to initiate the actual session, which contains one video stream.The states and adaptation paths have been negotiated during thepre-negotiation phase 106 just described. The QoS-Aware Application Unit202 of the Initiator 101 requests a start of the stream given a StreamIDand its desired StateID from the local Coordinator Unit 204, which firstperforms local admission tests and reserves the resources locallyrequesting the resources from the Local Resource Management Units 206.If the desired state cannot be fulfilled due to local resource problems,the Coordinator Unit 204 would request the FSM Engine 203 to try thenext QoS State having lower resource requirements according to thenegotiated Adaptation Path. After local successful resource reservation,the start request is propagated to the local Session Protocol Unit 205,which sends a START message to the peer's Session Protocol Unit 205message, which can be based on SIP extensions. This message contains theenforced StateID and Stream ID (which both can be modeled by using SDPextensions. The start message is forwarded via the peers CoordinatorUnit 204 to the peers FSM Engine Unit 203, which checks for a validStateID. If this is approved, the FSM Engine requests resources forhandling the media streams using the Coordination Unit 204 and theservices provided by the Local Resource Management Units 206, whichperform admission tests and reserve the resources at the peer.

[0223] Note that if the resources at the peer cannot be granted, thepeer Coordination Unit 204 requests the FSM Engine Unit 203 to select anew StateID according to the pre-negotiated Adaptation Path. Ifeverything is approved, the peer Coordination Unit 204 requests the peerSession Protocol Unit 205 to send an START_ACK to the Initiator's 101Session Protocol Unit 205 with the new StateID, which is forwarded viathe local Coordination Unit 204 to the local FSM Engine Unit 203. Thelocal FSM Engine Unit 203 checks for compatibility of the states and, ifapproved tells the local Coordination Unit 204 to request reservation ofnetwork resources (e.g. using RSVP) via the local Network ResourceReservation Unit 207. If the StateID were different from the originallyselected one, it would instruct the Local Resource Management Units 206to update the local resource reservation according to the new StateID(which is not shown in FIG. 2, because we assumed that everything wasfine). Additionally, the Coordination Unit 204 may request the SessionProtocol Unit 205 to send a RESERVE message to the peer 103, should thatpeer be responsible to manage network resource reservation within theaccess networks they are attached to [Marsh00]. In such a case, uponreception of the RESERVE message through its Session Protocol Unit 205,the peer 103 would trigger its Coordination Unit 204 to request networkresources via the peer's Network Resource Reservation Unit 207 (e.g.using RSVP). Once resources have been reserved at the network level, theCoordination Unit 204 at the peer 216 sends a confirmation back to thesender using the CONFIRM message, which is received by the local SessionProtocol Unit 205. This unit generates a confirmation to the localCoordination Unit 204. Once the Initiator 101 has collected all theconfirmations from all the Responders 103, it releases the role of theInitiator 101 and everything is setup properly.

[0224] For performing concurrency control, anybody starting aPre-Negotiation phase 106, Multi-Stream QoS Synchronization and QoSCorrelation phase 107, Fast negotiation phase 108, or Re-Negotiationphase 109 can take the role of Initiator 101. Therefore it will startsending messages to known peers inviting them to participate in thegiven phase. If none of the other peers is acting as Initiator 101, Peer216 realizes it is acting as an Initiator 101 and waits confirmation ofresource reservation (aggregate local and network) from Peers acting asResponders 103. Otherwise, if one of the Peers 216 is already acting asInitiator 101, said Initiator 101 will add the prospecting Peer 216 toits list of Responders 103, and the prospecting Peer 216 will wait toget invited by said Initiator 101.

[0225] Once the given phase is over, i.e. once the given Initiator 101has collected all the confirmations from all the Responders 103, thegiven peer acting as Initiator 101 “releases” the role of initiator. Inthis mode a token passing mechanism is used, where the acquisition ofthe token grants the role of Initiator 101 and the passing is done oncea phase is completed. In this way also the case of collisions is coveredby using a master-slave approach, where the master is the Initiator 101and the slave is the Responder 103. The Initiator 101 commands theResponder 103 when to start network reservation. In any case, networkreservation would be carried out concurrently among peers (except forthe case of a QoS reservation signaling protocol—like RSVP—whereavailable end-to-end).

[0226] Aspects of the Present Invention

[0227] Now the different concepts proposed by the present invention willbe explained:

[0228] Integration and extensions of different existing protocols andtechnologies for defining the E2ENP (including E2ERP): Morespecifically, this aspect encompasses the following phases: ProtocolDiscovery 104, Pre-Negotiation 106, Multi-Stream QoS Synchronization andQoS Correlation 107, Fast-Negotiation (with Economy Principle) 108,Re-Negotiation (with Economy Principle) 109, Resource ReservationRelease 110. All the six phases can be concatenated, or be executed atdifferent times. The Multi-Stream QoS Synchronization and QoSCorrelation 107 phase is optional, insofar as it is required only ifPeers 216 interact by sending/receiving multiple streams, which need tobe correlated and synchronized. The Protocol Discovery 104 andPre-Negotiation 106 phases can be executed a priori, and the results canthen be applied to multiple successive communication sessions, wherebyeach communication session is initiated with a specific Multi-Stream QoSSynchronization and QoS Correlation phase 107. Should even the resultsof the latter phase be applicable to multiple successive communicationsessions, each of said communication sessions can be initiated with aspecific Fast Negotiation phase 108. The protocol interacts with theLocal resource management units 206 during Pre-Negotiation 106,Multi-Stream QoS Synchronization and QoS Correlation 107, FastNegotiation 108, Re-Negotiation 109 and Resource Release 110 phases.

[0229] Economy Principle: According to this aspect, the resourceadmission control and resource reservation can be applied following thisorder: On the E2ENP Initiator 101 first, then on the E2ENP Responder(s)103, and finally on the network using 207. In the first two steps,issues like real-time CPU scheduling, memory management, and powermanagement are addressed. The same process applies in case ofre-negotiations, with the difference that previous allocated localresources need to be updated (either released or re-allocated) eitherimmediately or later during the Resource Reservation Release 110 phase.For the Resource Reservation Release 110 phase, first network resources,then peer, and finally local resources are

[0230] Concept of E2ERP: According tot his aspect, during runtime of aso established multimedia session, at any time any component may requestan adaptation. This again requires coordination of local, peer andnetwork resource management, according to the Economy Principle. Whenperforming this adaptation, user-defined or user-provided preferencescan be used implicitly in order to build the Adaptation Path.

[0231] Pre-negotiation of type of E2ENP: This can be done duringProtocol Discovery phase 104, either by forcing peers 101, 103 to querya Directory Server 102, or by having the latter announcing suchinformation. Instead of a directory server, a SIP registry server canalso be used.

[0232] A Pre-negotiation of capabilites: This can be done duringPre-Negotiation phase 106, by using specific protocol, e.g. SIP/SDP plusextensions.

[0233] A Pre-negotiation of complete codec list: This can be done duringPre-Negotiation phase 106, by using specific protocol, e.g. SIP/SDP plusextensions.

[0234] Pre-negotiation of Adaptation Paths at stream level: This can bedone during Pre-Negotiation phase 106, by using specific protocol, e.g.SIP/SDP plus extensions.

[0235] Pre-negotiation of Adaptation Paths at stream aggregation level:This can be done during Multi-Stream QoS Synchronization and QoSCorrelation phase 107, e.g. SIP/SDP plus extensions.

[0236] Indexing of pre-negotiated QoS Contracts and Capabilities forspeeding up the Fast Negotiation phase 108.

[0237] Indexing of pre-negotiated QoS Contracts and Capabilities forspeeding up the Re-Negotiation phase 109.

[0238] Modeling the data model, described in EPA 00 126 975.2, in anextended version of either SDP or SDPnG. Handlinginstallation/de-installation of Capabilities even at runtime, byexchanging asynchronous messages among peers for notifying such events.This can be based on e.g. SIP/SDP plus extensions.

[0239] Support of a E2ENP Broker 105: this is an optional third-partyentity, which can be used for relieving peers 101, 103 from performingtime- and resource-consuming Pre-Negotiation phase 106 (and eventuallyalso the Multi-Stream QoS Synchronization and QoS Correlation 107). Eachpeer 101, 103, after discovering from a Directory Server 102 which E2ENPBroker 105 to contact, can upload its Adaptation Path, and the list ofpeers to contact on said E2ENP Broker 105. The E2ENP Broker 105 may beimplemented e.g. as an enhanced version of audio-/videoconferencebridges, or by combining an enhanced SIP proxy server (or SIP redirectserver)—dealing with the negotiation process—and an enhanced version ofa SIP registrar—storing the Adaptation Path and Capabilities and all theintermediate negotiated versions thereof. The E2ENP Broker 105 thuscollects such information from each peers 101, 103, and performs thenegotiation process locally in its memory space. During this process,the E2ENP Broker 105 may eventually contact some of the indicatedresponders 103 on behalf of the initiator(s) 101, should such responders103 do not have stored their information on said E2ENP Broker 105. Oncethe negotiation has been carried out successfully, the E2ENP Broker 105will notify the peers 101, 103 about the availability of the results,and each peer can then download the results.

[0240] Use of E2ENP Broker 105 to allow multiple negotiation iterations,which would otherwise be too expensive. Responders 103 may disagree witha bid from the Initiator 101, and therefore provide-counteroffers. Theanalysis of counteroffers involves multiple round-trip messaging amongpeers, and therefore is too time consuming. The E2ENP Broker 105 canmediate and speed up the given process.

[0241] Extension of SIP and RTSP for Pre-Negotiation 106 of AdaptationPath (at stream level) and Capabilities, by piggybacking thisinformation in existing messages of SIP or RTSP.

[0242] Extension of SIP and RTSP for Pre-Negotiation of Adaptation Pathat stream aggregation level 107, by piggybacking this information inexisting SIP or RTSP messages (according to EPA 00 111 191.3).

[0243] Protocol procedures and messages for addressing Fast Negotiation108 and Re-Negotiation 109 phases (by using the identifiers associatedwith the pre-negotiated QoS Contracts of the Adaptation Path).

[0244] Extension of SIP/SDP and RTSP, by defining messages foraddressing:

[0245] a. QoS Contract negotiation;

[0246] b. coordination of network resource reservation (if any) amongpeers.

[0247] Extension of SIP and RTSP for Fast Negotiation 108, by usingindexed QoS Contracts and Capabilities.

[0248] Extension of SIP and RTSP for Re-Negotiation 109, by usingindexed QoS Contracts and Capabilities.

[0249] Coordination of network resource reservation mechanisms amongpeers, according to the Economy Principle, by introducing acknowledgedsynchronization procedures in SIP and RTSP during Fast Negotiation 108and Re-Negotiation 109 phases.

[0250] Extension of SAP for allowing Directory Servers 105 announcingtypes of end-to-end QoS negotiation protocols used.

[0251] Extension of existing directory protocols or of SIP with respectto the use of SIP Registrars, in order to store and/or retrieve types ofend-to-end QoS negotiation protocols used for enabling the ProtocolDiscovery Phase 104.

[0252] Extension of SAP for announcing Adaptation Paths with respect toDistribution Services, where AP are simply listing various multicastgroups, each targeting a specific QoS Contract.

[0253] All the E2ENP-related changes to SIP can theoretically be mappedonto H.323 as well.

[0254] Management of resource reservation release 110

[0255] The negotiation and re-negotiation of capabilities—during,respectively, the Fast-negotiation (with Ecoomy principle) phase 108 andthe re-negotiation (with Economy principle) phase 109—include thesignaling that the peers 216 need to exchange with each other, in orderto agree on the choice of audio and/or video codec and the configurationthereof (bitrate etc.)

[0256] Embodiment for the Implementation of the Present Invention:

[0257] Now an embodiment for the implementation of the present inventionwill be explained with reference to FIG. 2:

[0258] Each Peer 216 consists of a Computing Unit 201 containing aProtocol Stack 208 (e.g. IP and TCP/UDP) and a Coordination Unit 204.The Coordination Unit 204 coordinates.

[0259] 1. Protocol Discovery phase 104 through interface 214 to ProtocolDiscovery Unit 213;

[0260] 2. Pre-Negotiation phase 106 and Multi-Stream QoS Synchronizationand Correlation 107 through interface 209 to FSM Engine Unit 203, andthrough interface 210 to Session Protocol Unit 205;

[0261] 3. Fast Negotiation phase 108 through interface 211 to LocalResource Management Units 206, through interface 210 to Session ProtocolUnit 205, and through interface 212 to Network Resource Reservation Unit207;

[0262] 4. Re-Negotiation phase 109 through interface 209 to FSM EngineUnit 203 (for evaluating alternative QoS Contracts by using theAdaptation Path), through interface 211 to Local Resource ManagementUnits 206, through interface 210 to Session Protocol Unit 205, andthrough interface 212 to Network Resource Reservation Unit 207;

[0263] 5. Release phase 110 through interface 211 to Local ResourceManagement Units 206, through interface 210 to Session Protocol Unit205, and through interface 212 to Network Resource Reservation Unit 207.

[0264] A FSM Engine Unit 203 can be part of a given QoS-AwareApplication Unit 202, or can be a separate entity (middleware), whichmultiple given QoS-Aware Application Units 202 can use. The FSM EngineUnit 203 is a software process that, after having been configured with agiven Adaptation Path, is capable of implementing the FSM associatedwith the given Adaptation Path, on behalf of the QoS-Aware ApplicationUnit 202, in cooperation with the Coordination Unit 204.

[0265] A Protocol Discovery Unit 213 allows the Coordination Unit 204contacting a Directory Server 215 (which might also be implemented as anextension of a SIP Registrar), in order to accomplish the ProtocolDiscovery phase 104. The Protocol Discovery Unit 213 extends the use ofexisting specific protocols (e.g. LDAP, SAP, SIP), by advertising andquerying information about E2ENP, on behalf of the Coordination Unit204.

[0266] A Session Protocol Unit 205 allows the Coordination Unit 204carrying out the phases 106-110 with Peers 216. The Session ProtocolUnit 205 extends current protocols like SIP/SDP or RTSP (or evenH.323/H.245), by using SDP or SDPnG enhancements for describingbids/counteroffers for Adaptation Path and Capabilities, and byintroducing SIP or RTSP (or even H.323/H.245) specific messages forimplementing the signaling required for the phases 106 to 110,especially for the Pre-Negotiation phase 106 and the Multi-Stream QoSSynchronization and Correlation phase 107 (including the resourcecoordination according to the Economy Principle).

[0267] Interfaces 211 to the Local Resource Management Units 206 areidentified with respect to the following classes of resources:

[0268] CPU resources

[0269] Memory (primary and secondary) resources

[0270] Battery power

[0271] External auxiliary resources

[0272] For each class of resources, a specific Local Resource ManagementUnits 206 is made available, e.g. as an extension of the OperatingSystem loaded onto the Computing Device 201. The interfaces 211 allowthe Coordination Unit 204 to carry out the following tasks:

[0273] Monitor resource usage, according to various criteria (e.g.ordered per stream or per process)

[0274] Register (and de-register) for notifications to be forwarded tothe Coordination Unit 204 once and/or anytime a given condition is met

[0275] Reserve (and release) a given amount of resources (by e.g.setting a CPU deadline or pinning a memory area to a given process)

[0276] Update a given reservation (for the re-negotiate case)

[0277] Associate (and de-associate) a given entity, e.g. a process, to aclass of local resources usage (this is typically done for thoseresources that cannot be directly manipulate without OS supervision).The class is a kind of priority, which the OS can use for optimizingresource usage according to the user's policies.

[0278] Each Peer 216 can take the role of Initiator 101, provided thatall other peers are not acting as Initiator 101 in the meanwhile. Therole is maintained for the whole duration of any of the followingphases:

[0279] Pre-Negotiation 106,

[0280] Multi-Stream QoS Synchronization and QoS Correlation 107,

[0281] Fast-Negotiation (with Economy Principle) 108, and

[0282] Re-Negotiation (with Economy Principle) 109.

[0283] During the Resource Release phase 110, some of the peers notinvolved in the session tear down process—and thus willing to continuethe session with any remaining peer—may also take the role of theInitiator 101 in a concurrent phase 107, 108 or 109.

[0284] Should a Peer 216 (thereinafter the peer A) try to act as anInitiator 101 and invite other Peers 216 to act as Responders 103 forone of the aforementioned phases, while said phase is ongoing (i.e.while one of the other Peers 216 is already acting as an Initiator 101),the peer A would be notified that it needs to wait an invitation fromthe currently active Initiator 101, and the latter would have to addpeer A to its list of Responders 103.

[0285] The conclusion of any of the aforementioned phases coincides withthe collection at Initiator 101 side of all the confirmations ofsuccessfully reserved network resources from all the Responders 103. TheInitiator 101 has to validate each confirmation against to all theothers already received, in order to eventually rebalance the use ofresources among peers, by taking corrective actions.

[0286] Once the conclusion has been reached, the Peer 216 acting asInitiator enters in a neutral state, thus releasing the role ofInitiator 103 to any prospective Peer 216. The role of Initiator 103 canbe thus modeled as a logical token (the Initiator Role Token IRT), whichcan be exchanged among peers once any of the aforementioned phases isover.

[0287] Other Peers 216 can join an already active communication sessionamong multiple Peers 216. Is it really necessary and/or feasible inthese cases to re-negotiate everything? One key point of this inventionis the de-coupling of the Pre-Negotiation 106 phase from the actualMulti-Stream QoS Synchronization and QoS Correlation 107,Fast-Negotiation (with Economy Principle) 108, and Re-Negotiation (withEconomy Principle) 109 phases. As long as Peers 216 agree beforehand, atsession advertisement time, on what level of QoS to use, they can thenjoin (or leave) an active session anytime, by using the fast indexednegotiation process described in this invention. The key problem,however, is how to deal with Peers 216 joining the conference at a latertime, without having pre-negotiated anything. In this case, severaloptions are available:

[0288] 1. To re-negotiate everything from scratch among all Peers216—this is an unfeasible solution, not only because of the servicedisruption with high latency, but also because this solution wouldenforce dependencies among the connections belonging to e.g. avideoconference session, which is unrealistic in most of the cases.

[0289] 2. To force the new Peers 216 to accept the alreadypre-negotiated information: If a new Peer 216 does not accept this,he/she can not join the given session.

[0290] 3. To enforce media adaptation through some filters embedded intothe network, to meet the other preferences of the new Peer 216. Moredetails on this concept can be found in [Pasqu92], [Pasqu93], [Yeado93],[Kassl99a]) and [Kassl99b].

[0291] As a conclusion, this invention assumes that either solution 2 or3 is enforced by any application enabled to handle these issues (e.g. aconference control tool), and using the services of this invention thismeans that the solutions 2 and 3 are outside the scope of thisinvention.

[0292] Should the E2ENP Broker 105 be used and/or the negotiatedvocabulary be stored in the Directory Server 102 or SIP registrar, thenew participants can more advantageously get directly the pre-negotiatedinformation from such entities, and thus trigger only theFast-Negotiation (with Economy Principle) 108, Re-Negotiation (withEconomy Principle) 109.

[0293] The temporary asymmetric Initiator 101/Responder 103 scheme isequivalent to a master/slave configuration, where the Initiator 101 actsas a master, and each Responder 103 as a slave.

[0294] During one of the Pre-Negotiation 106, Multi-Stream QoSSynchronization and QoS Correlation 107, Fast-Negotiation (with EconomyPrinciple) 108, and Re-Negotiation (with Economy Principle) 109 phasesit is therefore possible to resolve possible conflicts arising fromPeers 216 independently initiating a phase.

[0295] During the normal conversation phase (i.e. after theFast-Negotiation 108 phase), multiple Peers 216 can detect any QoSChange/QoS Violation simultaneously and this can lead to collisions inrequesting to start correspondingly the Re-Negotiation 109 phase. Tothis extent, the last Peer 216 that took the role of Initiator 101 shalltake again such a role and arbitrate the requests among peers. Thismeans that each Peer 216 shall retain the identity of the Peer 216 thatlast acted as Initiator 101.

[0296] As a worst case, should however such a Peer 216 have left in themeanwhile the communication session, the collision would be inevitable.In such a case, each Peer 216 trying to “seize” the IRT, upon receptionof similar request from other Peers 216 shall block for a random time,and retry when such a time has elapsed. If during this time, anotherPeer 216 has sent a request to seize the IRT, the Peer 216 suspended onsaid timer would be notified accordingly, and consequently it shallabort both the timer and its request to seize the IRT, and consequentlytake the role of Responder 103. In FIG. 5 an example for theimplementation of the E2ENP Broker concept is shown, in which theSession Initiation Protocol (SIP) is applied as already mentioned inother parts of this application. SIP Proxy Servers and/or SIP RedirectServers 501 are associated with SIP Registrars 503. This inventionproposes to augment the location information 505 stored in the databases504 used by the SIP Registrars 503 by including the QoS Specificationand capabilities information 506 described in this invention. As well asthe negotiated QoS information 507 (both the temporary results beingdrawn during negotiation/re-negotiation phases, and the ones finallyagreed upon). Whenever registering with the SIP Registrar 503, users canthus upload the information 506 and 507. The E2ENP Broker 502 is a unitwhich communicates with the SIP Proxy Servers and/or SIP RedirectServers 501 over a specific interface 508 by means of well knownmechanisms (e.g. a JAIN SIP Servlet API): Whenever a E2ENP protocolimplemented as a SIP extension is used, the E2ENP Broker 502 performsthe negotiation and re-negotiation phases described in this invention inits memory space on behalf of all the peers by using the aforementionedinformation stored in the Registrar's database. The E2ENP Broker 502communicates with the SIP Registrar 503 over the interface 509.

[0297] The Main Advantageous Differences Between the Invention and theState of the Art Technologies Comment [Bor00] The hereby-proposedmechanism for managing conference state during sessions and findingappropriate configurations, by negotiating for configurations andchanging them, are based on this state of the art. This inventionintegrates this concept with local, peer and network resourcereservation according to the economy principle. The invention introducesadditional Adaptation Paths and QoS Contracts and references these usinghandles. Also, this invention includes terminal capabilities and networkresource reservation support into the negotiation mechanisms. [Burn00],The hereby-proposed mechanism to integrate local and [Kass100] andremote as well as network resource management in a [Kass100a] frameworkis based on this state of the art. In addition, this invention definesthe protocol that is necessary for negotiation of capabilities,preferences and QoS states and also identifies different versions of theprotocol using a directory server approach or SIP registrar. [Nahr95]The hereby-proposed economy principle is based on this state of the art.This invention integrates this concept with existing Internet protocols,includes Adaptation Paths and Indexes and considers other resources likebattery power or wireless sub- network availability. This inventionextends the work by first pre-negotiating several candidate QoSContracts and configurations thus building a common vocabulary.[RFC2533] The hereby-proposed mechanism to represent media handlingtogether with capabilities, express media feature sets and match featuresets [CONNEG00], (compare the bid with the local capabilities) is basedon this [CONNEG00a] or state of the art. This invention integrates thisconcept with [SDPCN01] existing Internet protocols, includes AdaptationPaths and integrates local as well as network and peer resourcereservation. This invention extends the work by first pre- negotiatingseveral candidate QoS Contracts and configurations thus building acommon vocabulary. [RFC2703] The hereby-proposed mechanism for protocolindependent negotiation of QoS contracts and capabilities is based onthis state of the art. This invention however negotiates QoS contracts,adaptation paths and capabilities instead of content as in [RFC2703].This invention also integrates this concept with existing Internetprotocols, includes Adaptation Paths and integrates local as well asnetwork resource reservation. This invention extends the work by firstpre-negotiating several candidate QoS Contracts and configurations thusbuilding a common vocabulary [SDPNG00] The mechanism proposed in thisinvention to perform together with endpoint capability negotiation andinclude user preferences [SDPCN00], and network resource reservation isbased on this state of the [Ott99], art. [SDPCN00] adds SDP extensionsfor capability [Beser00] negotiation, only. [Ott99] only provides anotation for configuration description. [Beser00] only provides SDPextensions necessary for endpoints to negotiate codecs. This inventionadds the negotiation protocol to determine a common set of configurationand integrates local, peer and network resource reservation by observingthe Economy Principle. In addition, the invention integrates the processof referencing configurations by handles and builds upon QoS contracts.Also, this invention not only negotiates codecs, but also terminalcapabilities and network resource reservation mechanisms. [SIPRES01],The hereby-proposed invention is based on the integration of togetherwith SIP/SDP with network resource reservation as a precondition.[Cama00], This invention extends the work by first pre-negotiating[Cama00a] several candidate QoS Contracts and configurations thusbuilding a common vocabulary. Also, only references to theconfiguration/QoS contract are referred to later on. In addition, thisinvention integrates local and peer resource management with the conceptprovided by this state of the art by observing the Economy Principle.This state of the art only covers one-to-one communication with resourcereservation and does not cover joining operations performed duringongoing sessions with negotiations. [Levin01] The hereby-proposedinvention fulfills many of the requirements stated in this state of theart. In addition, this invention provides the protocols and themechanisms to integrate local, remote and network resource managementinto a coherent framework observing the Economy Principle. [Levin01]only provides requirements and no protocol or mechanisms to enforce therequirements.

1. Method for providing guaranteed end-to-end quality of multimediaapplications and services, based on integration and coordination oflocal, peer, and network resource management, wherein peers (216)pre-negotiate (106) a common set of capabilities, qualities andadaptation mechanism before the actual communication takes place. 2.Method according to claim 1, comprising the step of pre-negotiating(106) a set of quality-of-service contracts and capabilities betweenpeers.
 3. Method according to anyone of the preceding claims, comprisingthe step of negotiating (107) quality-of-service correlation andsynchronization aspects among multiple streams among multiple peers. 4.Method according to anyone of the preceding claims, characterized inthat in case of a re-negotiation (109) during streaming the peers (216)refer to a pre-negotiated state, wherein said state refers to a given(pre-negotiated) quality-of-service contract and a given(pre-negotiated) set of capabilities.
 5. Method according to anyone ofthe preceding steps, characterized in that peers (216) negotiate duringthe pre-negotiation step (106), at the finest level of resolution, setsof quality-of-service contracts on a per-stream basis and/or on aper-stream-association basis, where stream associations are bundles ofstreams from one sender peer to a receiver peer.
 6. Method according toanyone of the preceding step, characterized in that peers (216) areinformed about changes in capability configuration.
 7. Method accordingto anyone of the preceding claims, wherein pre-negotiated alternativequality and configuration information of a given type of stream mayalready be available at a server for clients to choose from.
 8. Methodaccording to anyone of the preceding claims, comprising the steps ofProtocol Discovery (104), Pre-Negotiation (106), optional Multi-StreamQoS Synchronization and QoS Correlation (107), Fast-Negotiation (withEconomy Principle) (108), Re-Negotiation (with Economy Principle) (109)and Resource Reservation Release (110).
 9. Method according to claim 8,wherein all the six phases are concatenated and can be executed eithercontinuously or at different times (but still strictly following theorder indicated in claim 8).
 10. Method according to claim 9, whereinthe Multi-Stream QoS Synchronization and QoS Correlation (107) phase isoptional and required only if an Initiator (101) communicate withmultiple peers (216) by using multiple streams, which need to becorrelated and synchronized, based on user policies to be enforced atthe Initiator (101) side only.
 11. Method according to claim 10, whereinthe Protocol Discovery (104) and Pre-Negotiation (106) phases areexecuted a priori, and the results can then be applied to multiplesuccessive communication sessions, whereby each communication session isinitiated with a specific optional Multi-Stream QoS Synchronization andQoS Correlation phase (107).
 12. Method according to claim 11, wherein,if the results of the Multi-Stream QoS Synchronization andQoS-Correlation phase are applicable to multiple successivecommunication sessions, each of said communication sessions can beinitiated with a specific Fast Negotiation phase (108).
 13. Methodaccording to anyone of claims 8 to 12, wherein the protocol interactswith Local resource management units (206) during Pre-Negotiation (106),Multi-Stream QoS Synchronization and QoS Correlation (107), FastNegotiation (108), Re-Negotiation (109) and Resource Release (110)phases.
 14. Method according to anyone of claims 8 to 13, characterizedby a resource admission control and resource reservation appliedaccording to the following order: at the initiator (101) first, then atthe responder(s) (103), and finally for the network using (207). 15.Method according to claim 14, wherein in the first two said steps localresource management tasks like real-time CPU scheduling, memorymanagement, and power management are addressed.
 16. Method according toanyone of claims 8 to 15, wherein for the resource release (110), firstnetwork resources, then peer (216) and finally local resources arereleased.
 17. Method according to anyone of claims 8 to 16, whereinduring runtime of a so established multimedia session, at any time anycomponent of any peer (216) may request an adaptation, thus eventuallytriggering a Re-Negotiation phase (109).
 18. Method according to anyoneof claims 8 to 17, comprising the step of pre-negotiation of the type ofE2ENP during Protocol Discovery phase (104), either by forcing peers(101, 103) to query a Directory Server (102) which may be implemented asa SIP registrar, or by having the peers announcing such information. 19.Method according to claim 18, comprising the step of Pre-Negotiation ofcapabilities during Pre-Negotiation phase (106).
 20. Method according toanyone of claims 8 to 19, comprising the step of pre-negotiation of acomplete codec list during the pre-negotiation phase (106).
 21. Methodaccording to anyone of claims 8 to 20, comprising the step ofpre-negotiation of adaptation paths at stream level during thePre-Negotiation phase (106).
 22. Method according to anyone of claims 8to 21, comprising the step of pre-negotiation of Adaptation Paths atstream aggregation level during Multi-Stream QoS Synchronization and QoSCorrelation phase (107).
 23. Method according to anyone of claims 8 to22, characterized by the step of indexing pre-negotiated QoS Contractsand Capabilities for speeding up the Fast Negotiation phase (108). 24.Method according to anyone of claims 8 to 23, characterized by the stepof indexing pre-negotiated QoS Contracts and Capabilities for speedingup the Re-Negotiation phase (109).
 25. Method according to anyone ofclaims 8 to 24, characterized by the step of handlinginstallation/deinstallation of Capabilities even at runtime, byexchanging asynchronous messages among peers for notifying such events.26. End-to-End negotiation protocol for telecommunications,characterized in that a QoS-enabled communication session is establishedby pre-negotiating alternative QoS aspects and capabilities on anend-to-end basis to establish beforehand a common level of alternativeQoS and capabilities, the use of which all peers of thetelecommunication session can agree upon.
 27. Broker forEnd-to-End-Negotiation of quality-of-service, relieving peers of anetwork from carrying out the Pre-Negotiation phase (106) and,optionally, the Multi-Stream QoS Synchronization and QoS Correlationphase (107).
 28. Computer program implementing a method according toanyone of the preceding claims when run on a computer.
 29. Peer,configured for implementing a method according to anyone of thepreceding claims, comprising a coordination unit (204) coordinating thedifferent phases of the negotiation process of the distributed resourcemanagement process.
 30. Peer according to claim 29, characterized inthat the coordination unit (204) commands a Protocol Discovery (104),and then triggers and coordinates Pre-Negotiation (16), optionalMulti-Stream-QoS Synchronization and QoS Correlation (107),Fast-Negotiation (108) with Economy Principle, Re-Negotiation (109) withEconomy Principle, and Resource Reservation Release (110) phase. 31.Peer according to claim 29 or 30, characterized by a protocol discoveryunit (213) for advertising and querying information about the end-to-endnegotiation protocol to be used.
 32. Peer according to anyone of claims29 to 31, characterized by a session protocol unit (205) allowing thecoordination unit (204) to carry out the different phases of thecoordination process with other peers (216).
 33. Peer according toanyone of claims 29 to 32, characterized by interfaces (211) connectingthe coordination unit (204) to local resource management units (206).34. Peer according to anyone of the claims 29 to 32, characterized byinterfaces (214) connecting the coordination unit (204) to the protocoldiscovery unit (213).
 35. Peer according to anyone of the claims 29 to32, characterized by interfaces (210) connecting the coordination unit(204) to the session protocol unit (205).
 36. Peer according to anyoneof the claims 29 to 32, characterized by interfaces (209) connecting thecoordination unit (204) to a FSM Engine Unit (203).
 37. Peer accordingto anyone of the claims 29 to 32, characterized by interfaces (209)connecting the coordination unit (204) to a network resource reservationunit (207).
 38. Protocol according to anyone of the claims 25 and 26,characterized in that the negotiation and re-negotiation of capabilitiesinclude signaling of the selected codecs and the configurations thereof.